docs(site): make the llms.txt artifacts discoverable, and fix the landing-card parser - #23336

llms.txt and llms-full.txt are published but nothing points at them: they
are static files, so Docusaurus never routes them, the default sitemap
omits them, and no page links to them. A crawler or agent can only reach
them by guessing the path, which in practice means they are never found —
a browsing model asked one flag question read ~20 GitHub issue threads
instead, none of which are authoritative.

Advertise them three ways:

- two <link rel="alternate" type="text/plain"> head tags, so every page
  declares where the machine-readable copies live
- createSitemapItems, appending both URLs to the generated sitemap. The
  sibling ignorePatterns/lastmod options are closure-bound inside
  defaultCreateSitemapItems, so appended items are neither filtered nor
  double-processed and /search stays excluded
- a reader-facing section on the MCP page, with a pointer from
  "Why using Erigon?"

The section goes on the MCP page rather than "Why using Erigon?" because
the latter is a card-grid landing page, whose body generate-llms.py
replaces with synthesized bullets — prose added there would render on the
site but never reach llms-full.txt.

robots.txt is left alone: there is no standard directive for advertising
llms.txt, and the Sitemap: line already there now leads to both files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dx34ND1m4ySTJXqMR8kRDX

… claims

Review feedback from @yperbasis and Copilot on #23336. Five fixes.

1. Advertise llms.txt with rel="describedby", not rel="alternate".
   The llmstxt.org proposal defines describedby for the llms.txt file that
   covers a page, and reserves alternate + text/markdown for a *per-page*
   Markdown representation. A site-wide index is not an alternate
   representation of every page, and v2-aware agents look for describedby.

   llms-full.txt is no longer advertised in head at all: it describes no
   single page. It stays discoverable through llms.txt, the sitemap, and
   the MCP docs page.

2. Close a hole in the card-count guard. `if not cards: return None` ran
   before the count was taken, so a grid where *every* card failed to
   parse was indistinguishable from an ordinary prose page: the caller
   fell back to strip_mdx and the guard never ran — silently degrading
   the exact case it exists to catch. Count first, parse second, and
   return None only when the page has no cards at all.

3. Stop claiming llms-full.txt holds "every documentation page, in full".
   It does not: synthesize_landing replaces the whole body of a card-grid
   page with its card list, so why-using-erigon loses its introduction and
   prose. Describe the corpus as cleaned rather than verbatim, and say
   what is dropped.

4. Stop grouping llms.txt with llms-full.txt as "the whole documentation
   as one plain-text file" on why-using-erigon. llms.txt is only an index.

5. Drop "nothing else links to them" from the config comment — this PR
   adds the MCP page links, which makes it false.

Also refresh the stated file size, 420 KB -> 430 KB.

…, scope

Follow-up review from @yperbasis on #23336. Three findings, all confirmed
by reproduction before fixing.

1. The guard counted the marker it was validating. `expected` came from
   `lp-card-title`, so if a card lost or renamed that marker the count
   shrank in step with the loss it was meant to detect: verified that a
   two-card grid with one renamed marker emitted one bullet and raised
   nothing. If every marker changed, `expected` hit zero and the page
   fell back to strip_mdx. Count `lp-card` containers instead — the
   wrapper is not consumed by the parse, so the two signals stay
   independent. Attributes are now matched order-independently and `to=`
   is read separately, which the container match no longer pins down.

2. llms.txt had no route to llms-full.txt. Neither committed index
   contained the string at all, so once the llms-full head tag was
   removed, an agent following rel="describedby" reached an index with no
   way to find the full corpus. The generator now emits that link, and
   both copies are regenerated.

3. "Every documentation page" was still wrong. SECTIONS scans only docs/
   and help-center/, while the site also publishes v3.3 and v3.4 from
   versioned_docs/ — neither artifact contains those URLs. Say current
   documentation, and state the exclusion outright.

Two new tests: a renamed title marker must raise rather than be absorbed
(verified to fail against the previous count), and card attributes must
parse in either order.